Measuring The Cost Of A Cache Miss

It is vital that the cost of a cache miss be accurately measured in order for many hardware and software optimizations to occur. In this paper we describe a new technique, called pipeline spectroscopy, that allows pipeline delays to be monitored and analyzed in detail. We apply this technique to produce a cache miss ‘spectrogram’, which represents a precise readout showing a detailed histogram (visualization) of the cost of each cache miss. Cache miss spectrograms are produced by comparing instruction sequences and execution times that occurred near a miss in a 'finite cache' simulation run to the same set of instructions and execution times in an 'infinite cache' run. Cache misses are divided into clusters, and the miss penalty associated with each cluster is determined by identifying an upper and lower bound instruction around each miss cluster and calculating the cycle difference between these bounds. Detailed analysis of a spectrogram leads to much greater insight in pipeline dynamics, including effects due to miss cluster, miss overlap, prefetching, and miss queueing delays.